1. Field of the Invention
This invention relates to computer software and, more particularly, to the use of proxies in network traffic flow.
Portions of the disclosure of this patent document may contain material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure as it appears in the Patent and Trademark Office file or records, but otherwise reserves all copyright rights whatsoever. Sun, Sun Microsystems, the Sun logo, Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International in the United States and other countries. Products bearing SPARC trademarks are based upon an architecture developed by Sun Microsystems, Inc.
2. Background Art
A distributed system typically distributes functionality (e.g., computing or other services, storage, input/output, etc.) across multiple locations (or network nodes). The Internet is an example of a global-scale, highly available distributed system that interconnects computing networks and computer systems that can provide functionality. In a distributed system, information (e.g., text, graphics, sound, image, etc.) may be obtained from a variety of sources. It is becoming increasingly easier for computer system users to become interconnected. As more and more users are added to the Internet, for example, there is an increase in the amount of information that is transmitted via the Internet. The information may need to be created or modified (or customized) before it is transmitted to its destination. Such information creation, customization and dissemination cannot scale with today""s networking technology.
A proxy is a computer system (or software that executes on a computer system) that can provide various functionality. A proxy has been used to reduce the amount of information that must be transmitted via a network. Initially, a proxy was used to communicate across (xe2x80x9cget overxe2x80x9d) a firewall (i.e., a mechanism used to protect information in internal computer networks from external access). Proxies have since been used to cache (or store) information. In so doing, the information may be locally stored and available to each user of the internal computer network. The cached information was available to service multiple requests without requiring that it first be obtained from an external source thereby reducing the amount of network traffic.
A proxy can be used to provide other functionality such as content transformation (e.g., compression, decompression, encryption, decryption and reformatting), controlled external access to corporate intranets (sun.net reverse proxy), and for use in advertising and marketing on the Internet.
Thus, proxies can be adapted to provide additional and/or special-purpose functionality that can be used to meet the increasing need for information of interconnected computer systems and their users. However, there is a need to be able to ensure that the information flows through the desired proxy or proxies.
To better understand the need for ensuring the use of proxies, the following discussion of networks, the Internet and related topics is provided.
Networks
In modern computing environments, it is commonplace to employ multiple computers or workstations linked together in a network to communicate between, and share data with, network users. A network also may include resources, such as printers, modems, file servers, etc., and may also include services, such as electronic mail.
A network can be a small system that is physically connected by cables (a local area network or xe2x80x9cLANxe2x80x9d), or several separate networks can be connected together to form a larger network (a wide area network or xe2x80x9cWANxe2x80x9d). Other types of networks include the Internet, tel-com networks, the World Wide Web, intranets, extranets, wireless networks, and other networks over which electronic, digital, and/or analog data may be communicated.
Computer systems sometimes rely on a server computer system to provide information to requesting computers on a network. When there are a large number of requesting computers, it may be necessary to have more than one server computer system to handle the requests.
The Internet
The Internet is a worldwide network of interconnected computers. An Internet client accesses a computer on the network via an Internet provider. An Internet provider is an organization that provides a client (e.g., an individual or other organization) with access to the Internet (via analog telephone line or Integrated Services Digital Network line, for example). A client can, for example, read information from, download a file from or send an electronic mail message to another computer/client using the Internet.
To retrieve a file or service on the Internet, a client must search for the file or service, make a connection to the computer on which the file or service is stored, and download the file or service. Each of these steps may involve a separate application and access to multiple, dissimilar computer systems. The World Wide Web (WWW) was developed to provide a simpler, more uniform means for accessing information on the Internet.
The components of the WWW include browser software, network links, servers, and WWW protocols. The browser software, or browser, is a user-friendly interface (i.e., front-end) that simplifies access to the Internet. A browser allows a client to communicate a request without having to learn a complicated command syntax, for example. A browser typically provides a graphical user interface (GUI) for displaying information and receiving input. Examples of browsers currently available include Mosaic, Netscape Navigator and Communicator, Microsoft Internet Explorer, and Cello.
Information servers maintain the information on the WWW and are capable of processing a client request. Hypertext Transport Protocol (HTTP) is the standard protocol for communication with an information server on the WWW. HTTP has communication methods that allow clients to request data from a server and send information to the server.
To submit a request, the client contacts the HTTP server and transmits the request to the HTTP server. The request contains the communication method requested for the transaction (e.g., GET, PUT or POST). A GET method is a request may be used to retrieve a file or other information. A PUT method request is commonly used to store information that is contained in the request. A POST method may be used to process information in some manner (e.g., operate on the information contained in the request by a software program). The HTTP server responds to the client by sending a response (e.g., the requested information). The connection is terminated between the client and the HTTP server once the transaction is complete.
A client request therefore, consists of establishing a connection between the client and the HTTP server, performing the request, and terminating the connection. The HTTP server does not retain any information about the request after the connection has been terminated. HTTP is, therefore, a stateless protocol. That is, a client can make several requests of an HTTP server, but each individual request is treated independent of any other request. The server has no recollection of any previous request.
Instead of transmitting the information from the server that maintains the information, some systems utilize what is referred to as a proxy. Referring to FIG. 1A, proxy 102 acts as an intermediary between client 100 and server 104. Request 108 may be sent from client 100 to server 104 via path 118. However, instead of sending request 108 to server 104 via path 118, request 108 may be transmitted to proxy 102 via path 122. Proxy 102 may have the ability to carry out the request and return a response to client 100. If proxy 102 is not capable of replying to request 108, it forwards request 108 to server 104 via path 124. Proxy 102 may be configured to modify request 108 (e.g., reformatting, translating or transforming the some or all of the information contained in request 108) before forwarding it to server 104.
Similarly, instead of sending response 110 directly to client 100 via path 120, server 104 can send response 110 to client 100 via traffic path 126. Proxy 102 forwards response 110 via traffic path 128 to client 100. Proxy 102 may retain copies of documents or information fetched by request 108 for some time so that they can be accessed more quickly in the future, speeding up access for commonly requested information. This maintaining of information and fetched documents by proxy 102 is referred to as caching and the information maintained in the proxy 102 is referred to as a cache or proxy cache.
In the example of FIG. 1A, a single proxy is illustrated. It is possible to have multiple proxies or other types of intermediaries (e.g., a tunnel, a server such a gateway, etc.) between client 100 and server 104. A tunnel acts as a blind relay between two connections (e.g., client 100 and server 104). A gateway is typically used to connect two or more networks.
Because a gateway is used to connect networks, it sometimes includes or is used in conjunction with a firewall. A firewall is a mechanism that is used to protect information in internal computer networks from external access by blocking access between the client and the server. To provide limited access to information, a proxy or proxy server may sit atop a firewall and act as a conduit, providing a specific connection for each network connection. Proxy software retains the ability to communicate with external sources, yet is trusted to communicate with the internal network. For example, proxy software may require a username and password to access certain sections of the internal network and completely block other sections from any external access.
An addressing scheme is employed to identify Internet resources (e.g., HTTP server, file or program). This addressing scheme is called Uniform Resource Locator (URL). A URL contains the protocol to use when accessing the server (e.g., HTTP), the Internet domain name of the site on which the server is running, the port number of the server, and the location of the resource in the file structure of the server.
The WWW uses a concept known as hypertext. Hypertext provides the ability to create links within a document to move directly to other information. To activate the link, it is only necessary to click on the hypertext link (e.g., a word or phrase). The hypertext link can be to information stored on a different site than the one that supplied the current information. A URL is associated with the link to identify the location of the additional information. When the link is activated, the client""s browser uses the link to access the data at the site specified in the URL.
If the client request is for a file, the HTTP server locates the file and sends it to the client. An HTTP server also has the ability to delegate work to gateway programs. The Common Gateway Interface (CGI) specification defines a mechanism by which HTTP servers communicate with gateway programs. A gateway program is referenced using a URL. The HTTP server activates the program specified in the URL and uses CGI mechanisms to pass program data sent by the client to the gateway program. Data is passed from the server to the gateway program via command-line arguments, standard input, or environment variables. The gateway program processes the data and returns its response to the server using CGI (via standard input, for example). The server forwards the data to the client using the HTTP.
A browser displays information to a client/user as pages or documents (referred to as xe2x80x9cweb pagesxe2x80x9d or xe2x80x9cweb sitesxe2x80x9d). A language is used to define the format for a page to be displayed in the WWW. One example of a language that may be used to define a page is called Hypertext Markup Language (HTML). Other examples of languages include Standard Generalized Markup Language (xe2x80x9cSGMLxe2x80x9d) and Extensible Markup Language (xe2x80x9cXMLxe2x80x9d). To illustrate with reference to HTML, a WWW page is transmitted to a client as an HTML document. The browser executing at the client parses the document and displays a page based on the information in the HTML document.
HTML is a structural language that is comprised of HTML elements that are nested within each other. An HTML document is a text file in which certain strings of characters, called tags, mark regions of the document and assign special meaning to them. These regions are called HTML elements. Each element has a name, or tag. An element can have attributes that specify properties of the element. Blocks or components include unordered list, text boxes, check boxes, and radio buttons, for example. Each block has properties such as name, type, and value. The following provides an example of the structure of an HTML document:
 less than HTML greater than 
 less than HEAD greater than 
. . . element(s) valid in the document head
 less than /HEAD greater than 
 less than BODY greater than 
. . . element(s) valid in the document body
 less than /BODY greater than 
 less than /HTML greater than 
Each HTML element is delimited by the pair of characters xe2x80x9c less than xe2x80x9d and xe2x80x9c greater than xe2x80x9d. The name of the HTML element is contained within the delimiting characters. The combination of the name and delimiting characters is referred to as a marker, or tag. Each element is identified by its marker. In most cases, each element has a start and ending marker. The ending marker is identified by the inclusion of an another character, xe2x80x9c/xe2x80x9d that follows the xe2x80x9c less than xe2x80x9d character.
HTML is a hierarchical language. With the exception of the HTML element, all other elements are contained within another element. The HTML element encompasses the entire document. It identifies the enclosed text as an HTML document. The HEAD element is contained within the HTML element and includes information about the HTML document. The BODY element is contained within the HTML. The BODY element contains all of the text and other information to be displayed. Other HTML elements are described in HTML reference manuals.
In the example of FIG. 1A, request 108 and response 110 are transmitted between client 100 and 104 via proxy 102. By specifying a static set of preferences, a browser that is running on client 100 may be configured to direct request 108 to proxy 102. That is, the browser""s preference may be set to direct a type (e.g., a given communications protocol such as HTTP) of requests to a specific proxy. If the browser""s preference setting identifies proxy 102 as the proxy for the type of request 108, for example, request 108 is directed to proxy 102. This mechanism allows a user to statically set a preference for a proxy either by identifying a proxy""s URL or a URL of a configuration file that contains a proxy""s URL. A network address associated with a proxy may be obtained using a proxy advertisement mechanism (e.g., SLP or Web Proxy Auto Discovery (WPAD)). To change a proxy, the user must change the proxy designation in the browser""s preference or the configuration file. There is no ability to dynamically set a proxy based on each request that is generated by the browser, for example.
As an alternative to statically identifying a proxy in a browser""s preference settings, HTTP (i.e., HTTP version 1.1, or HTTP/1.1, discussed in Request For Comments (RFC) 2616) provides a redirection mechanism that allows server 104 to redirect request 108 to proxy 102 and requires the requester to repeat each request twice to complete the redirection. Referring to FIG. 1B, client 100 send request 108 to server 104. Server 104 transmits a response that contains a redirection status code and identifies proxy 102. In RFC 2616 (see section 10.3.6), the status code is referred to as a xe2x80x9c305 Use Proxy.xe2x80x9d
Upon receipt of request redirection 112 (i.e., status code 305), client 100 resends request 108 to proxy 102. Proxy 102 process request 108 as described above (e.g., provides a response, forwards request 108 to server, etc.). A disadvantage of this approach is that client 100 must send request 108 at least twice, once to server 104 and then to proxy 102. Further, the redirection mechanism is only available for use by server 104. Therefore, client 100 is dependent on server 104 to provide redirection information which may not be possible if server 104 is unavailable (e.g., behind a firewall or not operational). In addition, the redirection mechanism in HTTP can only be used to redirect a single request. That is, client 100 must access server 104 to obtain redirection information for subsequent requests.
A mechanism is needed to dynamically insert proxies or intermediaries (e.g., gateway, tunnel, server, etc.) in a network traffic path.
In embodiments of the invention, a method and apparatus for dynamic proxy insertion in network traffic path is described. According to one or more embodiments of the invention, a request and/or response message may be modified to identify a network (or traffic) node (e.g., a proxy, server, or intermediary). For example, a request directed to a server or a response directed to a client may be altered to insert a plurality of intermediate or final destination designations. In so doing, a path of a request or response may be altered dynamically.
In one or more embodiments of the invention, a thru-proxy tag is inserted in a response or request message to identify a network node or location. A response message may be modified by a receiving node to add a thru-proxy tag. If a response message that contains a thru-proxy tag is received by a client, the client retains the proxy identifier for use with subsequent requests. A request that is directed to an origin server that contains a thru-proxy tag is sent to the network node or location that is identified in the tag prior to sending the request to the origin server. A request may be modified by a receiving node to add a thru-proxy tag. The node that is identified in a thru-proxy tag may delete the tag from either the request or the response.
When generating a request, a client determines whether a request is related to a previous response and whether there is a thru-proxy tag associated with previous response. If so, the client adds the thru-proxy tag to the request. A request may be related to a previous response, if the request is initiated from a Web page sent in the previous response, for example. For example, a request for a resource identified by a hyperlink in a Web page is related to the response that contained the Web page. A request may be related to a previous response where the request is directed to the same location that provided the previous response. These are examples of the relationships that may exist between a response and a request. Other types of relationships may be used with one or more embodiments of the invention.
Since embodiments of the invention allow dynamic proxy designation, it is possible to specify a proxy for a given situation (e.g., a given request or resource). The dynamic proxy designation of one or more embodiments of the invention may specify a proxy for inclusion in more than one request. Further, embodiments of the invention provide a mechanism to designate a proxy at different levels (e.g. a default and overriding designations) and for multiple protocols.